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ABSTRACT 

In this paper constant feedback control laws are investi- 
gated for linear sampled-data systems. Derivations for the 
minimum-time problem and the infinite stage regular problem 
which yield constant feedback optimal control laws are 
presented. A case study for the finite stage regulator 
problem indicates that constant feedback control, although 
suboptimal, may be feasible from a design viewpoint. Several 
numerical examples illustrate the results. 
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1 . INTRODUCTION 



1.1 General 

Optimum performance implies that a system performs in 
the "best" possible manner according to some specified set 
of criteria. If a criterion is established in a mathematical 
form, called a performance index, the "best" performance results 
when this index is minimized or maximized. An optimum control- 
ler then generates the sequence of control signals necessary 
to obtain an extremum of the performance index. 

This thesis investigates the optimal synthesis of linear 
sampled-data control systems in order to determine an optimum 
or "suboptimum" control law in the form of stationary, linear 
feedback of the plant variables. 

If the performance index is specified as a quadratic 
functional on the plant variables and the control variables, 
for a linear system, the optimum control law that is derived 
to obtain an extremum of this functional generally results in 
a linear, time-varying function of the plant variables, i.e., 
multiloop feedback. The physical realization of a time-vary- 
ing control law requires a digital computer, or some other 
time-varying device. If a stationary control law can be 
obtained, the complexity of design, weight and cost of the 
controller is reduced. This is the motivation for this 
investigation. 
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It will be shown in two cases that the optimum control law 
is stationary. In a third case the optimum control law is 
time-varying, but a case study shows that a stationary control 
law can be obtained that is "close" to optimum in performance. 
1.2 System Description 

The physical system to be controlled is called the plant, 
and is described by a set of linear differential equations 
with constant coefficients. It is controlled by certain 
physical quantities, called inputs or controls. The physical 
variables of the plant which can be measured directly are 
called outputs. 

By definition, the state of a dynamic system is the 
smallest collection of numbers which must be specified at time 
t=t Q , along with the control for time t>t 0 , in order to be 
able to predict the behavior of the system for any time t>t 0 . 
In explanation, suppose the plant, which is described by an 
nth order differential equation, is simulated by an analog 
computer. This requires exactly n integrators. The outputs 
x^,...,x n of the n integrators, measured at any instant of 
time, are the state of the plant at that instant. The state 
may be regarded as a vector (nxl matrix) and is written as 



x(t) 



x 1 (t) 



where 



the components 



x 



n 



x x (t) , . . . 



(t) 

, x^(t) are the 



stat e var i ables . 
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The plant can be described by the state variable form of 



the differential equation 



x(t) = Ax (t ) + Bu (t) 



(1-D 



and the output equation 



y_(t) = Cx ( t) ; 



(1-2) 



or, as used in this discussion, by a vector matrix difference 
equation, the discrete time solution of equation (1-1) 



Here u is an m dimensional vector (m< n) denoting the control 
inputs to the plant; y is a p dimensional vector (prf n) 



denoting the outputs of the plant; A and (p are constant (nxn) 
matrices; B and D are constant (nxm) matrices; C is a constant 
(pxn) matrix. 

The methods required for the evaluation of the transition 
matrix, 0 , and the matrix D, from the matrices A and B, and 
the development of equations (1-la) and (l-2a) from (1-1) and 
(1-2), are given in Appendix I. 

It is assumed that the control is exerted on the plant by 
means of a piecewise constant signal whose level can change 
only at a regularly spaced "sampling points" or "sampling 
instants" . Thus, 




(1-la) 



and the output equation 



y^(kT) = Cx(kT) . 



(l-2a) 
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u*(t) = u(kT) ; kT^ t -= (k+l)T 



k = 0,1,2 



t • • • 



(1-3) 



where T is a positive constant called the sampling period; 
t = kT is the kth sampling point. 

It is customary to symbolize the sampling operation by 
means of a special element, the sample and hold element (S.H.E.), 
which converts the continuous signal u(t) into a sampled 
signal u*(t) as given by equation (1-3). The difference equa- 
tion (1-la) relates the state x((k:+l)T) at the (k+1) st sampling 
point to the state x(kT) at the kth sampling point and to the 
control signal, ja(kT) . Fig. 1-1 is a block diagram representa- 
tion of equation (1-la) . The blocks, in general, represent 
matrix multiplication as indicated by the double lined arrows. 

Where convenient, the sampling period, T, will be dropped 
from the notation. It will be understood that the conversion 
from a specific sampling point to an instant in time will 
require the sampling period. Equation (1-la) and (l-2a) will 
now be written 



It should be noted that the physical quantities repre- 
sented by the state variables and output variables are contin- 
uous functions of time. The values of the functions at the 
sampling instant are x(k) , etc. For convenience and where no 
ambiguity results, the notations x(t) , x(k) , or simply x may 



x(k+l) 




+ D(T)u(k) 



(1-lb) 



y (k) = Cx(k) 



( l-2b) 
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be used interchangeably. This will hold also for command 
inputs, r(t) and i:(t) . 

To optimally control a plant governed by equation (1-la) 
it is desirable to express the control input u(k) as a function 
of the state and of all present and future desired states. 

The concept of "desired state" is a generalization of what 
is usually called "command input" or "desired output". If 
all of the state variables cannot be measured directly, and 
if the plant is "observable", it is possible to estimate the 
values of unmeasurable variables through use of the Kalman 
Filter ^ 1. , 2 J . It will be assumed that all of the state 
variables are available. 

In general terms, the design of optimum control involves 
the determination of a control law or a control sequence so 
that the control process can be taken from a given initial 
state, x(0) , to a desired state, x^# in some finite time. A 
"control law", u£x(k),kj, is the control signal expressed as 
a function of the state variables of the plant and possibly 
time. A "control sequence", ii(k) k=0,l,..., is the control 
signal expressed as a function of discrete sampling instants. 
1.3 The Performance Criterio n 

If a system is to be optimized, some criterion must be 
chosen that determines how well the system is operating. The 

criterion established depends upon the performance require- 
ments of the system. The method used here is to assign a 
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cost to the system at each sampling instant. The sum of 
these costs over the period 0^ t^ NT is called the performance 
index and one form is given by 

J N =£ fx T (k)Qx(k) + u T (k-l)Ru(k-l)l (1-4) 

k=l L 

where N is the number of sampling intervals to accomplish the 
desired state transition. The matrix Q and the matrix R are 
weighting factors which allow the relative cost of the errors 
and the control effort to be adjusted to achieve the desired 
performance. For optimal design, the performance index is to 
be minimized. The performance index will be discussed in 
detail in Chapter 3. 

1.4 The Problem Statement 

The problem can now be stated: Given the system defined 

by equation (1-la) , find the control sequence u(0) ,u(l) , . . . 

. .,u(N-l) such that for any initial state x(0) and some 
desired output, a performance index of the form given by 
equation (1-4) is minimized. Furthermore, determine if a 
stationary, linear feedback control law can be established 
such that the total cost over some period is optimum or "close" 
to optimum. 
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2 . 



TIME OPTIMAL OR DEADBEAT RESPONSE 



2 ■ 1 Problem De ginition 

It is desired that the system achieve equlibrium with 
zero steady-state error from any initial state as quickly 
as possible. In this and following sections, in the interest 
of simplicity, the desired state of the plant is identically 
zero for all t. It will also be assumed that the plant has a 
single input (m=l) , i.e„, u(k) is a scalar. 

In oftler words, the object of control is to transfer any 
initial state x(0) to the equilibrium state x(N)=0 in a 
minimum number of sampling periods. It is desired to find the 
control sequence, u(0), u (1) , . . . , u (N-l) , and the control law 
which will accomplish this. A derivation of the control 
sequence is given from which a geometric interpretation can 
be obtained. The result is a stationary control law. 

This problem may also be stated in terms of the performance 
index 

J N = x T ( n )Q^( n ) (2-1) 

where Q is the identity matrix and N indicates the final 
stage of the process. The problem then is to find the control 
sequence such that N is the smallest integer for which J^=0 
and x( t )~£ f° r t — NT. Chapter 3 will show a dynamic 

programming solution of this problem. 
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2.2 Controllability 



A plant is said to be completely controllable if there 
exists a control signal u(t), defined over a finite interval 



to a desired equilibrium state x(t^) in time interval t^-t 0 . 

In the present case the state x^l) is the state 0 . 

A necessary and sufficient condition for complete control- 
lability in the discrete-time case may be stated as follows: 

An nth order linear discrete-time process with a scalar 
control is completely controllable if, and only if, the 
vectors 



are linearly independent . How these vectors arise will be 
shown. 

Consider first a second-order system (n=2) . Any two 
values of the state variables x-^ (k) and x^ (k) define a point 
in the x-^, X 2 plane which is called the state space of the 
system. Analogously, any point in the state space is called 
a state. The evolution of the system from an arbitrary present 
state (x^(k), X 2 (k) ) can be represented by a curve in the state 
space, called the trajectory, along which time is a parameter; 



t ^ t< t^, which moves the process from initial state x(t Q ) 



s ± =(p (-T)d(T) 
-1 = 0 (-2T)d(T) 




(2-2) 
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Fig. 2-IA TRAJECTORY IN STATE 
SPACE 




Fig. 2-2 THE CANONICAL AND 
ORTHOGONAL CO-ORDINATE SYSTEMS 
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see Fig. 2-1 where the arrow indicates the positive direction 
of time. Since the solution of a linear homogeneous differ- 
ential equation, i.e., u(t)=»0, is uniquely determined for all 
finite values of time by the initial state, there is one, and 
only one, trajectory passing through any given point in the 
x^, X 2 plane. The behavior of the system may be thought of 
as the motion of a point along a trajectory. If the point 
does not move at all, the point is said to be an equilibrium 
state . 

If v x , v 2 are the unit vectors along the co-ordinate 
axes in the state space, then the state vector at any time 
kT can be written in the form (see Fig. 2-2) 

x(k) = x^(k)v-^ + x 2 (k)y. 2 (2-3) 

Now, from what initial states x(0) is it possible to reach 
the origin of the state space in one sampling interval? The 
transition equation, again, describing the process is 

x (k+1) =(p (T)x(k) + d (T) u(k) (2-4) 

For k=0, equation (2-4) yields 

x(D =(f) (T)x(O) +(T)u(0) (2-5) 

Thus, if it is assumed that x(l)=0» the initial state from 
which this equilibrium condition can be reached in one 
sampling period is, solving equation (2-5) for x(0) and using 
equation (2-2), 

x (0) = -0(-T)d(T)u(O) = -u(0)£ 1 (2-6) 
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(A property of the transition matrix is that (p (T) = 0(kT) , 
e.g.,0 _1 (T) = 0(-T) ; see Appendix I) . Equation (2-6) states 
that if u(0) is not bounded, all initial states x (0) lying 
along the vector s_^ can be taken to x(T)=0 in one sampling 
period. The states satisfying equation (2-6) constitute a 
straight line in the state space. 

The set of all states from which the origin can be 
reached in at most two steps is that set from which a state 
given by equation (2-6) can be reached in at most one step. 

For k=l, equation (2-4) yields 

x (0) = -u(O)0 (-T)d(T) - u(l)0 (-2T) d(T) 
or 

x(0) = -u(0)s^ - u(l)s ^2 (2-9) 

If the vectors s_^ and _s^ are linearly independent, i.e., 
they do not lie along the same line through the origin of the 
state space, they form a basis in the state space. That is, 
any other vector in the state space can be written as a 
linear combination of _s^ and s^ • 

It can be shown jl,5jthat the vectors 

s-l - 0(-T)d(T) 

s 2 = 0 (-2T)d(T) (2-10) 

are linearly independent if, and only if 

T u ¥ q rr (2-11) 

where q is any integer and iJ is the imaginary part of any of 
the complex poles of the transfer function of the plant. This 
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means that the input must excite all natural frequencies 
of the plant; in other words no cancellation of poles must 
be possible in the transfer function. In illustration 
consider a second-order process with the transfer function 



G(s) - ~ 2 T 

s *u 2 

From a flow graph of this process (see Chapter 3) , or from 
the state equations 



x 1 =(Jx 



2 



x 



2 



= -CJx 1 



and the methods in Appendix I, the following matrices result 



0(T) = 



cosCJT sinCu/T 
-sinC^T cosC^/T 



d( T ) = 



-cosC7t/CJ 
sinCJT /(J 



Using equation (2-10) 

(-cos(*7t cos2ijT - sinC^T sin2tjT) /(jJ 
(-cosCJt sin2L7T + sinCu/T cos2CJt) /(^J 
By inspection it is seen that if T(^/= qJC , where q is an 
integer, s_^ = +_ s^ an< ^ hence the two vectors are not linearly 
independent . 

If the values of the control signal, u(0) and u(l), in 
equation (2-9) are not bounded, all initial states x{0) 
lying in the plane containing vectors s and s^ can be taken 
to the equilibrium state x(2) = 0_ in two sampling periods. If 
the values of the control signal are bounded, it would take a 
longer time to reach the equilibrium state. 
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The vectors and s^ may be regarded as unit vectors 
along the axes of a new co-ordinate system, x(k) having the 
co-ordinates m (k) and m~ (k) in this new system, (see Fig. 

2-2) . This co-ordinate system will be called the canonical 
co-ordinate system. 

2.3 Determination of the Control Sequence and Control Law 
The next step is to determine the actual values of th<| 
sequence u(0), u(l), which will take an arbitrary ihitial 
state x(0) to the origin of the state space. 

It was stated in the previous section that every initial 
state can be written in the form: 

x(0) = m 1 (0)_s 1 + m 2 (0)_s 2 (2-12) 

where the itk 1 s are real numbers. If the process can be taken 
from the initial state x(0) to the equilibrium state in two 
sampling periods, it can be taken from the state x(l) to the 
equilibrium state in one sampling period. The state vector 
x(0) is a two-dimensional vector and the state vector 5£(1) 
is a one-dimensional vector in the canonical co-ordinate 
system. Then, x(l) may be expressed as 

x(l) =m- L (l)s^ (2-13) 

Equation (2-4) with k=0 is 

x(l) =(/>(T) x (0) + d (T) u (0) (2-14) 

Substituting equation (2-12) for x(°) • 

x(l) =0(T) [iti-lCO)^ + m 2 (0)s 2 j + d(T)u(0) (2-15) 
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Simplifying using equation (2-10) , the definitions of s^ and 
and properties of the transition matrix, it follows that 
x(l) = m 1 (O)0(T)^)(-T)d(T) + m 2 (O)0 (T)0 (-2T)d(T) +d(T)u(0) 

x(l) = m-jJOjdCT) + m 2 (0)s^ 1 + d(T)u(0) 
or 

x(l) = m 2 (0)s^ 1 + |m 1 (0) + u(0)J d(T) (2-16) 

Hence, x(l) will have the form required by equation (2-13) if. 



and only if, 

u (0) = -m 1 (0) (2-17) 

and 

m 2 (0) = m 1 (l) (2-18) 

If x(l) is to be reduced to zero in one step, then 

x (2) = 0 = 0(T)x(l) + d (T) u (1) (2-19) 

Substituting equation (2-13) for x(l) , 

x(2) =0(T)m 1 (l)s + d(T) u (1) (2-20) 

Using the definition s^ =0(-T)d(T), 

x(2) = U^l) + u(l)]d(T) (2-21) 



Therefore, the condition for x(l) to be reduced to zero in 
one step is 

u ( 1 ) = -m 1 (1) = -m 2 (0) (2-22) 

which is the desired control signal at t=T. Hence, at every 
sampling instant, the optimal control signal u(k) is given by 
u(k) = -m-^ (k) , (k=0,l for a second-order 

process) (2-23) 
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where in general for any state, 

x(k) = m 1 (k)s^ 1 + m 2 (k)s^ 2 , k=0,l (2-24) 

The effect of the control will now be determined. Substi- 
tuting equation (2-24) for x(k) into equation (2-4), and using 
the optimal control signal as given by equation (2-23), 
x (k+1) =Cp{T) [m 1 (k)£ 1 + m 2 (k)£ 2 j“ d(T)m 1 (k) 

Simplifying, using equation (2-10) , the definitions of s^ 
and s^, 

x(k+l) = m 1 (k)d(T) + m 2 (k)s^ - d(T)m^(k) 
x(k+l) = m 2 (k)^ (2-25) 

Comparing this with equation (2-24) , rewritten with k=k+l, 

x(k+l) = m^(k+l)_s^ + m 2 (k+l)s^ 2 (2-26) 

we see that after the application of the optimal control 

m-^(k+l) = m 2 (k) (2-27) 

In the general case for an nth order system, equations (2-24) 
and (2-25) are 

x(k) = m^(k)£ 1 + m 2 (k)s^> +...+ ^(k)^ (2-24a) 

and 

x(k+l) = m 2 (k)jS^ + m 2 (k)s^ +...+ m n (k)s^ n _ 1 (2-25a) 

and therefore 

m^ (k+1) = m i+1 (k) (2-27a) 

A geometric interpretation of equations (2-23 to 2-27 a) is 
as follows. The initial state of the plant may lie anywhere 
in the n-dimensional state space. After one sampling period 
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and the application of a control signal as given by equation 

(2-23) , the state can be expressed as a linear combination of 

only n-1 linearly independent vectors s,,...,s , ; in other 

“ 1 —n-1 

words, after one sampling period, the initial state always 
lies in an (n-1) -dimensional space. The initial state is 
taken successively into a space of one less dimension during 
each sampling period. Ultimately, the state must be in a 

t 

space of dimension zero which is an equilibrium point in the 
state space. 

For the second-order case, the initial state x(0) lies 
anywhere in the s^,s^ plane. After the application of one 
control, the state x(l) lies on the line described by the 
vector s^. The final control takes the state to the origin. 
2.4 Computing the Control Signal 

According to equation (2-23), the optimal control signal 
is obtained by computing the first component in the canonical 
co-ordinate system, m-^ (k) , of the state of the plant at each 
sampling instant. It will now be shown that the optimal 
control is given by 

u(k) = ^ T x(k) (2-28) 

where a is an nxl vector of constants, i.e., the control law 
is stationary. 

Any component of a vector in one co-ordinate system can 
be expressed as a linear combination of the components of 
the same vector in a different co-ordinate system. Thus, 
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it is always possible to write, considering the second- 
order process, 

m^(k) = a^x-^(k) + a 2 x 2 (k) (2-29) 

where the a^ are real constants (see Fig. 2-3). It is then 
necessary to calculate these constants to obtain the control. 
Observe that by definition, 

x(k) = x^(k)v^ + X 2 W— 2 (2-30) 

and 



x(k) = m^(k)^ + m 2 (k)s ^2 (2-31) 

In matrix notation, these expressions may be equated and 
written as 



[ 





x x (k) 
x 2 (k) 




m x (k) 
m 2 (k) 



By definition 



[ 




J0 (-T) d(T) I 0 (-2T) d(T)] 



p ll P 12 
_ P 21 P 22 




(2-32) 



(2-33) 



The basis vectors Sj,j3 2 are related to the basis vectors 
— 1 # — 2 by 



[— ii— 2] = 



p ll p 12 
P 21 p 22 



This equation states that 



si - Pn^i + p 2 i— 2 




s 2 = Pi 2—1 + P 2 2— 2 


(see Fig. 2-4) 



(2-34) 



(2-35) 
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Fig. 2-3 VECTOR COMPONENTS 




Fig. 2-4 COMPONENTS OF THE CANONICAL 
BASIS VECTORS IN THE ORTHOGONAL 
CO-ORDINATE SYSTEM 
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Substituting equation (2-34) in equation (2-32) , 



N 2 *] 



" x i ( k ) 
x 2 (k) 



[^uH [ p ] 



m 1 (k) 
m 2 ( k \ 



(2-36) 



Note that the vectors an< ^ Z 2 are un i t vectors in an 
orthogonal co-ordinate system, and hence, have the values 



l< 

II 


V 


II 

CM 

>1 


"o’ 




_o_ 




_1_ 



Therefore the matrix 

'l 0 



[sii ^2] 



.0 1 , 

is the identity matrix. Solving equation (2-36) , 



~m 1 (k)‘ 


■ M _1 


"x 1 (k)' 


_m 2 (k)_ 




- X 2 W - 



(2-37) 



or in general for an nth-order system. 



m (k) = P -1 x(k) 



(2-38) 



where P denotes the inverse of the matrix P =fs n ;s_! [si. 

If the system is completely controllable, the required inverse 
exists. Thus, equations (2-23) and (2-38) together define the 
control law u^x(k)j : Letting 



-1 



a ll a 12 



nl 



l ln 



nn 



u(k) = -m^ (k) = -a T x(k) 



(2-39) 

(2-40) 



where a 1 is the notation for a vector transpose and is the 

.-1 



first row of the matrix P 



The optimum control law is 
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therefore constant feedback of the state variables at sampling 
instants. Fig. 2-5 shows the block diagram for an arbitrary 
second-order process implementing the stationary feedback law 
when the desired state is Q. 

The control sequence can also be found as a function of 
the state x(0) . From equations (2-23) and (2-27a) , the 
following relations may be derived 

u (k) = -m 1 (k) = -m k+1 (0) (2-41) 

Equation (2-38) with k=0 is 

m (0) = P -1 x (0) (2-42) 



Hence, 



u = -P ^(O) 



(2-43) 



where 



u = 



u(0) 

u(l) 



u(n-l) 



(2-44) 

The optimal control was derived here for a second-order 
system but holds for an nth order system where the initial 
state vector can be expressed as. 

x(0) = -u(0)s^ - u(l)^ 2 u (n-l)£ n (2-45) 

If the n vectors 

s^ = 0(-iT)d(T), i = 1,2,... ,n (2-46) 

are linearly independent and if the values u(i), i=0,l,..., 
n-1, are not bounded, the initial state x(°) can be brought to 
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Fig. 2-5 OPTIMAL REGULATOR SYSTEM 



the equilibrium state x(N)=C) in at most n samples; that is, 
t^;f nt. The control law was shown to be stationary. For a 
derivation with vector inputs see |^4j . 

2.5 Arbitrary Polynomial Inputs 

The foregoing discussion shows the derivation of the time- 
optimal control law when the desired equilibrium point with 
zero error is the origin of the state space. The solution will 
now be extended for a follow-up system. This problem is 
equivalent to making the first component, y^ (k) , of the plant 
output vector 

£(k) = Cx(k) ( l-2a) 

equal to some desired output, r(t) or r(k), in such a way 
that the plant is in equilibrium with zero system error. 

If r(t) is an arbitrary function of time, the problem 
cannot be solved in general. However, when r(t) has the poly- 
nomial form 

A P " 1 

r(t) = g 1 + g 2 t+ ... + g p t ; t20 

= 0 ; t*= 0 (2-47) 

(where the g^ are arbitrary real numbers) the solution can be 
obtained by modifying the solution of the previous regulator 
problem. 

Note that the components of the plant output vector, > 

are the physical variables of the plant which can be measured 
directly. These components are linear combinations of the 
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plant state variables. For example may be the sum of the 
state variables and x 2 - It will be assumed at this point, 
for simplicity, that 



Yl = x 1 



(2-48) 

It is known that the optimal control requires knowledge 



of all of the plant state variables. It then seems feasible 
to derive from £(t) a command input state vector, _r(t) , to 
which it is desired to drive the plant state vector so that 
the plant is in equilibrium with zero system error. Then, 



from equation (2-47) , form 

r x (t) = r(t) = g 1 + g 2 t +...+ g p tP 
r o (t) = i|2L(iL = g 9 + 2g^t + ...+ (p-1) 



-P-2 



dt 



^ (p-l)£/.L.\ 

r - (t) = ~ Tp-if '' 1 = (p_1) (2-49) 



dt 



Also, note that 



agrtt i= o 



Then 



dt* 



r (t) = 



(2-50) 



r x (t) 



r p (t) (2-51) 

Note that equations (2-49) require knowledge of the derivatives 



of the input. This may be difficult to obtain in some cases. 

If the plant contains at least p "free" integrators (with- 
out feedback) , that is, the plant has a type number p, then 
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the differential equations of the plant in terms of the 



state variables 



x 

X 



1 

2 



are 



x. 



x^ - 
'2-.o 



X P X p+1 (2-52) 

plus other equations which do not involve the state variables 

x , ...,x n . Form the modified state variables 
1 ir 



A 

X 1 



A 



X 



2 



= X 1 - 



r 



1 



= x* 



- r , 



A 

X 



= x - r 
P P P 

A 

X = X ■ 

i i 



p -= n 

Substituting equation (2-53) into equation 



the modified state variables obey the same 



A 



tions. Hence, in equilibrium when x=0. 



X 1 “ r l 



X 2 = r 2 



(2-52) shows 
differential 



(2-53) 

that 

equa- 



x. = 0 , p«=i<n (2-54) 

which shows that the output will be identical with the input 
after, at most, n sampling periods. 
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Thus, the regulator system which contains at least p free 



integrators is converted to a follow-up system simply by 
taking the difference between the plant state x(k) and the 
input state r^(k) to create the modified state x prior to 
multiplying by the feedback coefficients a^ . In matrix 
notation, then, 

u (k) = -a T Jx(k) - r(k)j 



(2-55) 



(the components, r p+i * • • • > r n > of vector r_ will be zero). 

Fig. 2-6 shows the block diagram for a simple second- 
order plant with one free integrator and, therefore, the input 
r(t) = constant (p=l) . Fig. 2-7 shows the block diagram of 
a follow-up system for the general case where u (k) is given 
by equation (2-55) . 

The foregoing solution states that the plant must have 
at least p free integrators, that is, it must be at least of 
type number p in order to follow an input r(t) that satisfies 



the differential equation 

y\ 



dPg(t) = Q 



(2-50) 



dt* 



For example, a plant of type number 2 will follow only "step" 
and "ramp" inputs (pf 2) with zero steady-state error. However, 
this restriction on the allowable order of the input to a system 
with a given number of free integrators may be relaxed some- 
what as follows. 
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?(t)=c -1c, 




Fig. 2- 6 FOLLOW-UP SYSTEM EXAMPLE 




Fig. 2-7 VECTOR BLOCK DIAGRAM 
OF FOLLOW- UP SYSTEM 
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Consider the example of Fig. 2-6 but where the command 



input is a "unit ramp", 
r(t) = t 



or, using equations (2-49) 



r (t) 



t 



1 



(2-56) 



The desired response then is x-^(oo) = t, and x^(oo) = 1. 
Looking at a signal-flow graph representation of the system, 
Fig. 2-8 (disregarding R) , we note, however, that there is a 
feedback of x^ upon itself through an integrator and gain of 
-d. This means that could not possibly hold the steady- 
state value of 1. Stated another way 
x 2 (o° ) = u(co) -d • x 2 (co ) 

0 4 0 - d* (1) (2-57) 



We can get around this difficulty by introducing an auxiliary 
input, R, as shown in Fig. 2-8. Equation (2-57) could then be 
written 



) = R = d*x 2 (oo) 

0 = R - d- (1) (2-58) 

Therefore , 

R = d- (1) (2-59) 

It is seen, then, that through the use of an auxiliary input 
a plant can be made to follow a polynomial input of one order 
higher than the type number of the plant. The introduction of 



36 



R 




Fig. 2-8 FOLLOW-UP SYSTEM EXAMPLE 
WITH AUXILIARY INPUT 




Fig. 2-9 FOLLOW-UP SYSTEM EXAMPLE 
FOR ARBITRARY (IN A CLASS) INPUT 
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the auxiliary input simply results in a shift in the origin 
of the canonical co-ordinate system. The equilibrium state 
is still reached in k;f n stages. 

It is seen from equation (2-59) that, if the auxiliary 
input is to be implemented as shown in Fig. 2-8, the command 
input and its derivatives must be known beforehand and may not 
be arbitrary. However, by implementing R as a function of the 
input state vector _r, the system will follow an arbitrary, but 
still restricted, polynomial input. 

Continuing with the example of Fig. 2-8, it is seen 

that 

u = - an( x ! “ r x ) - a 12 ( x 2 “ r 2 ) + R (2-60) 

where 



R = d*r 2 (2-61) 

Then substituting for R, and rearranging, 

u = a il r l + ( a i 2 + d ) r 2 ~ a ll x l “ a 12 x 2 (2-62) 

If the system is redefined as shown in Fig. 2-9, and if 

b 1 = a^ and b 2 = ( a ^ 2 + d) , it is seen that u is still given 
by equation (2-62) . The input vector, r_, may now represent 
an arbitrary polynomial, in a class, and the origin of the 
canonical, co-ordinate system will be automatically shifted 
so that the equilibrium state may be reached. 

In general matrix notation the control, u(k), is given 



by 



u(k) = b T r(k) - a T x(k) 



(2-63) 
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where 



b T = fb b ...b b , ...bl (2-64) 

— L l 2 p p+1 nj 

(the elements b , n . . .b , where p«=n, are zero and must be 
p+l n 

included in the vectors b and r so that they will be conform- 
able with the (nxl) vectors a and x- ) Elaboration is required 
for the general formulation of the vector b. 

It can be shown ^6j that for a scalar control, u(t), and 
a completely controllable system, the coefficient matrix A 
of equation (1-1) can be written in a canonical form. This 
results in the signal-flow graph representation of any plant 
as shown generally in Fig. 2-10 (output not shown) . The 
plant shown has m free integrators. From the preceding example 
it can be seen that the elements of the vector b are formed as 
follows: 

b. = a,. + 0 ; i = 1, . . . ,m 
x lx 

b = a._ + 0 y p<m+l 
p IP 

b P = a iP + Vi -• p = m+1 

b. = 0 ; j = p+l,...,ny p*=n 

1 

where p is the order of the polynomial input and where d m+ ^ 
is shown in Fig. 2-10. The a^ are the optimal gains given 
by the first row of P 1 . 

It has been shown that a plant can be made to follow a 
polynomial input function of order one higher than the number 
of free integrators in the plant. This will be illustrated in 
Chapter 4. 
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The state of the plant, x(t) , is a continuous function 
of time and is being fed back continuously, while the control 
input is constant over the sampling period. Thus, if the 
desired value of any of the state variables is not a constant, 
and if it is being fed back to the input, then the error at 
the input will not be zero at all times and therefore the 
system will not be in equilibrium. This will also be illustrated 
in Chapter 4. 

If the plant output is other than that given by equation 
(2-48), y^ = x-^, the command input vector r_ must be written 
in a form to correspond with the desired values of the state 
variables. For example if the output of the second-order plant 
shown in Fig. 2-9 is given by 

Yl = X 1 + x 2 ' 

and the desired output is a ramp, 
r(t) = gt 

then for zero error 



y = gt = x 1 + x 2 (2-66) 

Taking the derivative, 
g = x-^ + x 2 
or 

g = x 2 + 0 ; x^ = x^ (2-67) 

for equilibrium. Solving equations (2-66) and (2-67) simultan- 
eously. 
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X x = g(t-l) = r 



( 2 - 68 ) 



x 2 = g = r 2 

Equation (2-68) gives the desired values of the state variables 
and, hence, the command input vector. 
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3. GENERALIZED PERFORMANCE INDICES AMD DYNAMIC PROGRAMMING 



3.1 General 

In this chapter dynamic programming will be applied to 
optimize the performance of a dynamic system. The results 
are applicable to both sampling and continuous systems, a 
fact which will be discussed later in the chapter. This 
approach leads to a stationary control law for the infinite 
stage regulator problem. 

3.2 Problem Definition and Development 

Consider the system discussed in Chapter 1 described 
by the difference equation 



A performance criterion is established to measure "how 
well" the system's output follows the command input r(t). 
This is called the performance index and is denoted by J. 
Fig. 3-1 shows schematically how the performance index is 
evaluated. 

Inputs ; The method of dynamic programming requires the 
knowledge of all the desired states; therefore, the system 
will be optimized for an expected class of command inputs 




(3-D 




A member r(t) of this class of inputs is defined by 



r(t) = r 1 (t) ; t ^ 0 



= 0 ; t«= 0 (3-2 



(3-2) 
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where 



£ (t) = (p (t)w(O) = 



(t) 



r (t) 
P 



(3-3) 



0 r‘ fc > is the transition matrix of a pth order linear differential 
equation with constant coefficients and w(0) is an arbitrary- 
vector. (f) determines the class of inputs for which the system 
is to be optimized and w(0) determines a particular member of 
that class. 

Example 1 : To illustrate the formulation of (^) , suppose a 

member of the class of inputs |r(t)^ satisfies the differential 



equation 



a 2 r(t) = o 
dt 2 



(3-4) 



The equations for the state variables, (2-49) , lead to the 
flowgraph shown in Fig. 3-2. (j) may be determined by inspec- 

tion of the flowgraph: 

0 r (t) 

(3-5) 



~l/s 


i / 2 1 

1/s 


= 


"l t“ 


_0 


1/s . 




_0 1 _ 



Then, from equation (3-3) , 

r^ (t) = w 1 (0) + tw 2 (0) 

Example 2 : The solution to the harmonic equation 

2 



(3-6) 



(3-7) 



W 1 + U w 2 = 0 ; t “° 

yields a sinusoidal signal. This equation may be decomposed 
into 
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Fig. 3-2 FLOWGRAPH FOR POLYNOMIAL 

EXAMPLE 




Fig. 3-3 FLOWGRAPH FOR SINUSOIDAL 

EXAMPLE 
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Vf ± =U™ 2 



w 2 = -Uw 1 (3-8) 

From the state equations or from the flowgraph shown in Fig. 
3-3, (p is seen to be 



Then 



0 r (t) =^* 1 



U 



2 , i2 2 I ,2 

s +LJ s +LJ 



-U 



2 , .2 2 , .2 

_s +(^7 s +c*y 



cosL^t sinCJt 
-sinLJt cosC^t 



r 1 (t) = w l (0)cosCJt + w 2 (0)sinLyt 



(3-9) 



(3-10) 



If a system has been designed for optimal performance for 
a particular class of inputs, it will approach nearly optimal 
performance with any other input signal which can be closely 
approximated by an r(t) over one sampling interval |^3j . 
Performance Index : The error between command input and plant 

output is given by 

e ( t) = r ( t) - y x (t) (3-11) 

The undesirability of error is not usually a function of the 
sign, so a more plausible measure would be 
e ( t) = | r ( t) - Yi (t)| 
or 

e ( t) = (r ( t) - y x (t) ) 2 (3-12) 
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The second is commonly used because of its mathematical 



convenience. A generalized measure of error is defined as the 



quadratic form 

_z T (t)Qz(t)- 0 (3-13) 

where Q is an (n+p) by (n+p) positive semi-definite matrix, 
and js(t) is an (n+p) by 1 vector defined by 



z (t) = 



’x(t) 

r(t) 



(3-14) 



Example : To illustrate the preceding development, suppose 

the desired measure of error is 
(r x (t) - x L (t) ) 2 
Then, the Q matrix is given by 



q l,l " q n+l,n+l " 1 
q l> n+1 q n+l,l 1 

all other elements = 0 (3-15) 



To complete the notation in terms of the combined state vector 
z^(t) , define 



D (T) = 


" D x< T >" 






D (T) 
L r 




<p( T) = 


0 X (T) 


1 

' 0 
1 




„ 0 


! 0 r (T) _ 



(3-16) 



(3-17) 
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where D (T) , (h (T) and (7) (T) are as previously defined, and 
x r x * r 

(T) is 0^ The latter follows from the definition of jr(t) 
in equation (3-3) where there is no D^_(T) term. 

The two state transition difference equations are com- 
bined as 

z^k+1) =(^)(T)z^k) + D(T)u(k) (3-18) 

In many systems the control energy represents cost and, 
therefore, it may be desired to consider control energy in the 
measure of performance as well as the system's errors. The 
quadratic form is also used for measuring control effort 
u T (t)Ru(t) 



As stated previously, if the performance of a system is 
to be optimized, a criterion must be selected to tell "how 
well" it is operating. This criterion is the performance 
index, which, over the period 0^ t^ NT is given by 

J N [z(°) 'H( T ) » • • wU( (N-l)T)] 

NT 



=J £,z T (t) Qz^( t) + u* T (t)Ru*(t)^ (t)dtj 



(3-19) 



0 



where Q and R are proportional weighting matrices which com- 
promise between minimization of errors and control energy. 

/^(t) is a positive time-variant weighting function. Note that 
the smaller J N the more satisfactory the system's performance. 

The foregoing definition is very general and by varying 
Q,R and A(t) the index is adjusted to fit problems of physical 
significance . 
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Example 1 : The response of a plant is never instantaneous. 



and as a result for some period of time following the applica- 
tion of the control signal, errors are inevitable. As time 
progresses, these errors are expected to become smaller. 
Therefore, it may be desirable to have A(t) increase with 
time. If the errors and the control effort are to be measured 
only at the sampling instants and given zero weight at all 
other times, then, 

J m = E [A< k > (z T (k)Q.z(k) + u T (k-1) Ru (k-1) )1 (3-20) 

k=l 

where A(k) is defined as the value of A(t) at the kth sampl- 
ing instant. If the weighting of errors and control effort 
remains the same at all sampling instants, then X (k) is 
dropped from the performance index. 

Example 2 : A performance index concerned only with the error 

at the Nth sampling instant is 

J = z T (N)Q£(N) (3-21) 

N 

If Q is the identity matrix, this index is the Euclidean 
norm of the system's state variables at the end of the Nth, 
or final stage of the process. Note that it reflects no 
concern whatever for what goes on in the early stages of the 
process, nor for the control effort required. It reflects 
the desire to bring the system as close as possible to the 
origin. 
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If a system has deadbeat response as described in Chapter 



2, it is possible to select a control signal that will make 
Jj^ = 0 for a finite N and any initial state z^(O) . 

3.3 Recurrence Relations £ 7 ] 

It is now possible to state the problem more precisely. 
Given a linear plant defined by equation (3-1) and a class of 
input signals defined by equations (3-2) and (3-3) , find the 
sampled control signal u(k) for the t^O which minimizes the 
performance index. That is, find the sequence of vectors 
u(0) , u(l) , u(2) , . . . such that for any initial state z^(O) , 
the performance index J^, equation (3-19), is minimized. The 
performance index considered is 




which is the case where ,\(k) is dropped from equation (3-20) . 



The optimum return function J°l^z(0) is defined as the 
minimum value of J and has the known simple form 





(3-23) 



which is proved by induction later. is a symmetric. 



positive semi-definite matrix. By definition 

min min 




N+l 
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z_(l) is determined solely by u(0) , which enables equation 
(3-24) to be written as 

min ( min 



J N+1 [z(°)] = u(0)<jz T (l)Qz(l) + u T (0)Ru(0) + u(l), 

min min N+l ) 

u(2) , . . . ,u(N) Y, [jE. (k)Qz(k) + u (k-l)Ru(k-l)J> 
k= 2 J 

(3-25) 



The last group of terms is exactly J°j^z(l)j which gives 





min [ 




= u(0) J 



(3-26) 



This equation can be arrived at directly using the 
principle of optimality, which states that the minimum cost 
of an N+l stage process is the minimum of the sum of the cost 
of the first stage and the minimum cost of the N remaining 
stages. Note that the arguments of the state and control 
variables increase with time, whereas the subscript on 
decreases with time. 

Substituting equations (3-18) and (3-23) into (3-26) 

J N + l[i< 0) ] = 

Klin f m 

= u(O)J[0z(O) + Du(0)] T (Q + P N ) [0z(O) + Du(0)J + u T (0)Ru(0) 

* (3-27) 

Completing the square on the right side of (3-27) and 

defining 



u ' (0) = A n+1 z(0) (3-28) 

Ajj+y = ~[d T (Q + P N )° + R; _1 D T (Q + P ^Xp (3-29) 
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Equation (3-27) becomes 



min t 



T, T 



j n +i[ 2(°)] = u(0)|[u(0) - u 1 (0)] |d (Q + P n )D + R ][u(0) - u' (0)] 



+ z T (O)0 T (Q + P n )0z(O) 



- i T (°)Vi[ DT<Q + V D + R ]Vi i (0) | (3 - 30) 



The control ]j( 0) occurs only in the first term. If the matrix 
^D T (Q + P^)D + r] is positive definite the optimal control is 
unique. Then, the minimum value of the first term is zero 
and occurs when 

u (0) = u' (0) (3-31) 

The matrix will be positive definite if R is positive definite, 
or if Q is positive definite and the columns of D are linearly 
independent. It will not be positive definite if R = 0, and 
the columns of D are linearly dependent. In other cases the 
matrix might be singular. 

Equations (3-31) , (3-28) and (3-29) define the optimal value 

of u(0) . 

The recurrence relation for P ^ is determined by equating 
(3-30) and (3-23) when u(0) = u' (0) , 

J N + 1 [- (0) ] = 2 T (O)0 T (q + P N )0z(O) 

- z T (0)A T n+1 [d T (Q + P n )D + r]a n+1 z(0) 

= z T (0)P n+1 z(0) (3-32) 
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Since equation (3-32) must hold for all z^(O) , the recurrence 
relation becomes 

P N+1 =0 T (2 + P N> «£ + da N+1> < 3 ‘ 33 > 



where 



aT N+i[ dT <2 + p N > d + R 



=0 t (q 



+ p n> d 



(3-34) 



or 



a n+i = ‘[ dT(q + p h )d + r ]‘ 1dT (Q + P H>0 



( 3-34a) 



has been used to simplify (3-33) . 



Equation (3-32) shows that if the quadratic form for j' 



N 



is correct, then has the same form. The quadratic form 

is trivially correct for Jq since 

j£[z(o)] = 0 



(3-35) 



for all z^(O) . To complete the mathematical induction, the form 
for J° must be shown to be correct. is determined from 

equation (3-32) and (3-29) noting that Pq = 0. Equation (3-34) 
is again used to simplify the result. 

J?[Z(0)] = z T (Q)0 T Q [0 - D(D T QD + R) _1 DQ0]z(O) 

= z T (0)P 1 z(0) (3-36) 

J 2 ^z( 0 )j has the desired quadratic form. 

3.4 Solution of the Problem 

The solution proceeds using the recurrence relations, 
equations (3-33) and (3-34a) . Since J^£jz(0)j =0, Pq = 0; 

calculate A-^ . From A-. and Pq calculate P-^. Continue this 
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process until all A of interest are calculated. If the 

N 

plant is controllable, A will tend to a limit as N increases. 

N 

Therefore, for the infinite stage regulator problem the 
optimal control takes the form of a stationary linear function 
of the states, which is independent of the sampling instants kT 
u(k) = A^z (k) (3-37) 

However, if it is desired to optimize over a finite number 
of samples, then the optimal control takes the form of a 
different linear function of the states at each sampling 
instant . 

u (k) = A(k)_z(k) (3-38) 

An advantage of this design procedure is that it always 
results in a stable control system. This is due to the state 
method of describing the system's dynamics, and to the use of 
dynamic programming to obtain the solution J^3 J . 

The resulting system is shown in Fig. 3-4. Note that 
it is a multiloop feedback system, requiring measurement of 
the plant's states as well as the input states. 

If T becomes small, relative to the plant's time constants, 
the sampled signal u*(t) is nearly the same as the continuous 
signal u(t) . Then the sample and hold element can be removed 
from the system, resulting in a continuous system. If R = 0, 
and if T approaches zero, then the control input becomes 
unbounded. For this reason R must be greater than zero if T 
is going to be very small, or if a continuous system is desired 
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A comment on notation is made here. When the control 



input u is a vector quantity the feedback control matrix will 

be denoted as A^, whereas, if the control input is a scalar 

quantity, the feedback control matrix will be denoted as 
T 
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4. NUMERICAL EXAMPLES 



To illustrate the control laws derived in Chapters 2 
and 3, the following examples are presented. All simula- 
tions were done with the CDC 1604 digital computer. 

4.1 Deadbeat Response 

The third order system shown in Fig. 4-1 was simulated. 
The descriptive equations are 

x(k+l) =0x(k) + du(k) 
u (k) = ajjx(k) 

The state transition matrix^), and the d matrix for a sampl- 
ing period T = 77/4 are 




d = 



1 .7091 

0 .7265 

0 -.6088 
.06965' 
.2509 
.5585 



.2509 

.5585 

.3914 



Using the deadbeat performance index 
J N = x T (N) I x(N) 

and the recurrence relations derived in Chapter 3, the 
following control law was computed, where, due to the use of 
a scalar control, N is equal to the order of the plant, 
u (k) = a^x(k) - ^ a 2 a 3 j x( k ) 

= '-2.741672 -3.127373 -2.045130 ■ x(k) 
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The same gain matrix was obtained using the methods of 
Chapter 2. The system's trajectory is presented in Table 4-1. 



Stage (k) 


Control u(k) 


x x (k) 


CM 

X 


x 3 (k> 


Initial (0) 


-2.7416 


1.000000 


0 


0 


(1) 


3.064975 


.808983 


-.687893 


-1.531268 


(2) 


-1.711838 


.150569 


-.585992 


1.531268 


Final (3) 


0 


0 


0 


" 0 



TABLE 4-1 SYSTEM TRAJECTORY 
FOR DEADBEAT EXAMPLE 



The recurrence relations also yield a series of gain 
matrices forming a control law which is a different function 
of the states at each sampling instant, 
u(k) = a.^(k)x(k) 

This control law gives the identical control sequence u(0), 
u ( 1) , . . . , u (N-l) as the stationary control law above. 

4.2 Infinite Stage Regulator 

The optimum control law for the infinite stage regulator 
is also in the form of a stationary linear function of the 
state variables. The control gains and the cost function can 
be computed by the recurrence relations of Chapter 3. After 
the optimum gains are established, the system's trajectory is 
computed from the difference equation of the system. 

This example utilizes the same third order plant as the 
deadbeat example. Fig. 4-1. The problem was formulated as 
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in Chapter 3 and the resulting system is as shown in Fig. 3-4. 
The performance index is 



J. 



N 




where 



Q 



100-10 



R = 1.0 



0100-1 



T = 77/4 



0 0 10 0 



10 0 10 



0-1001 



For a polynomial input such as described in equations (3-4) 
through (3-6) , the gain matrix was computed to be 

ajj = [-.581355 -.626587 -.922714 .581355 1.716587] 

The command inputs used were a unit step and a unit ramp, 
and the results are shown in Fig. 4-2 and 4-3 respectively. 

Both step and ramp responses were within 1 % of the desired 
values at the end of eleven samples. Note that the gain 
matrix has the pattern expected for a system with one free 
integrator as discussed in Chapter 2; i.e., that = -a 

and a 2 = ~( a 2 + d) = - ( a 2 ” 1*09) • In the development of 
Chapter 2, it was shown that such a system would reach equilib- 
rium with step and ramp command inputs only if in addition to 
the optimum plant state feedback gains, optimum gains on the 
command input states were incorporated; i.e., if the optimum 
gains are not applied to the input states, the system's ramp 
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20 . 30 . 40 . 



response will have a steady state error. A ramp command 
input was applied to the example's system, without the gains 
on the input states, and the response is shown in Fig. 4-4. 

For a sinusoidal command input as developed in equations 
(3-7) through (3-10) , the gain matrix was computed to be 
-N = [~* 581355 “♦ 626587 -.922714 .37783 .581434] 



Note that the first n elements are the same as for the poly- 
nomial command input, but and a n+ £ are different. The 

system's trajectory is shown in Fig. 4-5. 

Fig. 4-6 shows the trajectory for the same system with 
a sinusoidal input but with the weighting matrix Q as given 
by equation (3-15). The gain matrix was computed to be 

a T = [-4.484 -4.275 -2.313 4.080 1.8151 
— N l J 

4.3 The Finite Regulator Problem (CASE STUDY) 

As stated in Chapter 3, if it is desired to optimize 
a system's performance over a finite number of samples M, 
where M is either smaller or not much greater than the number 
of recurrence steps required for A^ to tend to a limit, the 
resulting control takes the form of a different linear func- 
tion of the states at each sampling instant. 



u(k) = A(k)x(k) 

A problem of this type would occur if the command input to the 
system were reinitialized every M samples. An example might 
be a missile that is using a search-while-track radar and is 



64 




65 





66 




67 




receiving new information at the rate MT, where T is the 



system's sampling period. 

To physically implement such a system would require a 
computer to reset the control feedback gains at each sampling 
instant. The engineering complexity involved with such a 
system warrants investigating a system with fixed control 
feedback gains. Though not optimal, this system might be 
considered superior due to other criteria such as weight, 
cost, etc. This form of control will be referred to as sub- 
optimal in this section. 

An investigation of suboptimal control was accomplished 
by simulating second, third and fourth order systems and 
comparing the performance to optimal. The performance index 
used was 



and most of the investigation was made with the second order 
system of Fig. 4-7. 



The state transition matrix (£), and the d matrix for a 
sampling period of 0.1 are 



J, 



N 




(p = .9952 .0931 

-.0931 .8636 



d 



004768 



0931 
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The weighting matrices are 




0 

1 




The system was run for five samples . 

It was observed that if the system was run for five samples 
using as the fixed control feedback matrix, the performance 
was not greatly degraded from that of optimal. This was also 
true of the higher order systems. The amount of performance 
degradation that is acceptable is, of course, an engineering 
decision based on the physical aspects of the problem. 

Runs were also made using the values that tended to 
in the limit. Even though these values gave optimal perform- 
ance in the infinite stage regulator problem, the total cost 
was higher in the five-stage finite regulator problem than 
that obtained using the gain matrix A^ . 

The elements of A^ were then perturbed to see if the 
performance could be improved. This led to an exhaustive 
search which resulted in a gain matrix which differed signifi- 
cantly from Acj , and the resulting performance was near optimal. 
During the search the system's parameters and the initial 
conditions were held constant. 

Using the gains obtained in the search, various initial 
conditions were tried, generally resulting in performance 
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close to that obtained with A^ . The results of this case 
study are presented in Table 4-2. The column A p contains 
the costs using gains obtained by perturbation. 



Initial 

Conditions 

x(0) 


5 

Total Cost J 5 =£ [x T (k)Ix(k) + u 2 (k-l)] 


Optimal 


Using 

A 5 


Using 
A 00 


Using 

a p 




’3' 

1 




48.658 


48.712 


55.159 


48.7714 




’l" 

1 


* 


8.0210 


8.0500 


9.5956 


8.0214 




’l" 

3 




28.0741 


28.2912 


34.2085 


28.1898 




10‘ 

.2 




488.0655 


488.2655 


531.8718 


490.7636 




.2' 

10 




230.8611 


233.1031 


270.0404 


233.5962 



TABLE 4-2 RESULTS OF CASE STUDY OF SUBOPTIMAL 
CONTROL FOR THE FINITE STAGE REGULATOR PROBLEM 

* Initial conditions used during exhaustive search 



Gain Matrices: 



A 5 


'1 


-.038279 


- . 26493lj 


A 


■ 1 


[-.376449 


- . 509776 j 


A 

P 




[ .045198 


-.302418] 


Optimal (in 


order of application) : 


A 5 




[-.038279 


-.264931] 


A 4 


_ 


[-.021133 


-.243223] 
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A 3 = [-.007999 -.216953] 

A 2 = [ .000587 -.183544] 

A x = [ .004439 -.139468] 
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5. CONCLUSIONS 



5.1 Summary 

5.1.1 Deadbeat Response 

The optimum control law derived for deadbeat response 
takes the form of a stationary linear function of the state 
variables, independent of the sampling instant kT: 
u(k) = A^c(k) 

where N is the number of stages required to bring the system 
to the origin, N< n. 

It has also been shown that by using dynamic program- 
ming to minimize the performance index 
J = x T (N)Qx(N) 

where x(N) is the final stage vector and Q is the identity 
matrix, the resulting control sequence will give deadbeat 
performance. The control law may be either stationary or a 
different linear function of the states at each sampling 
instant kT. The control sequence is the same in both cases, 
u (k) = A^.x (k) = A (k) x(k) 

5.1.2 The Infinite Stage Regulator 

The optimal control law derived for the infinite 
stage regulator is in the form of a stationary linear func- 
tion of the state variables. The procedure for obtaining the 
constant control gains is well established J^3, '/j , and Is given 
in Chapter 3. 
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5,1.3 The Finite Stage Regulator 



The optimum control law derived for the finite 
stage regulator is a different linear function of the state 
variables at each sampling instant, 

;u(k) = A(k)j£(k) 

If it is desired to achieve the best performance 
possible with a stationary control law, the results of a 
case study tend to indicate that using A^ as the fixed gain 
matrix, where M is the number of sample periods, results in 
a system's performance not greatly degraded from optimum. 
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APPENDIX I 



Derivation of Difference Equations Representing 
a Sampled-Data Dynamic System 
A block diagram of the system discussed in this Appendix 
is shown in Fig. 1-1. 

Plant : A linear, time invariant dynamic system with state 

equations, which in vector notation may be written 

x(t) = Ax(t) + Bu(t) " (1-1) 

^(t) = Cx(t) + Du(t) (1-2) 

The Laplace transformation of equation (1-1) is 

sX(s) = x(O-) = AX( s) + BU( s) (1-3) 

Rearranging, 

sX (s ) - AX (s) = x (0-) + BU ( s) 

(si - A)X(s) = x(O-) + BU( s ) 

X(s) = (si - A)x(O-) + (si - A) _1 BU(s) (1-4) 

Let 

</)( s) i (si -A) -1 

4>w iX _:L [0( s )] 

The solution of (1-1) becomes 

(t-T)Bu(T)dT (1-5) 

0 

where it is assumed that u(t) = 0 ? t«= 0, and(^)(t) is causal. 
More generally: (initially t = t^ vice zero) 



x(t) =Cj)( t)x(O-) + J(p 
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x(t) = 0( t-t 0 )x(t 0 ) +[(p(t-T)Bu(T)aT 



(1-6) 



natural response forced response 
For an interval of time T set: 



t 0 = kT 



t = (k+l)T 
Then, (1-6) becomes: 

(k+1) T 

x ( (k+1) T) = 0(T)x(kT) +S 0 ( (k+l)T-T)Bu(T) d r 

kT 

(1-7) 

where u(t) is piecewise constant, i.e., 
u 1 (t) = (kT) ^ 



u 2 (t) = u 2 (kT) 



u ( t) = u (kT) 
m m 



\ for kT^t -=(k+l)T 



Then, 



(k+1) T 
kT 



J 0((k+l)T-T)Bu(r)dT =< J(fc(T-T) BdT 



u (kT) 



= D (T) u (kT) 



Then, (1-7) becomes 

x(k+l) T =0(T)x(kT) + D (T) u (kT) 

Some Properties of the State Transition Matrix 
1) 0(0) is an identity matrix 
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2 ) (p( kT) = «0( T)) k 

. ATlc AT 

Since by definition, 0(kT) = e = (e ) 

3 ) 0(-kT) = (0(kT)) _1 

“AkT AkT, -1 
Since(^) (-kT) = e = (e . ) 
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